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ABSTRACT 

This study investigated the effect of cohort participation in a learning community and 
collaborative learning techniques on the success of first-year engineering students. Student 
success was measured as gains in knowledge, skills, and attitudes, student engagement, and 
persistence in engineering. The study group was comprised of students registered in an 
engineering orientation course. Two sub-groups were compared: a) a test cohort and b) a control 
group. Students registered in the test cohort shared four courses. The control group shared only 
one course, an engineering orientation course. An exit survey was used to measure gains in skills 
and attitudes, student engagement, and student perceptions of collaborative learning techniques. 
Participation in the learning community test cohort was not found to improve student success. 
Participation in the learning community test cohort model was, however, linked to more positive 
student attitudes towards engineering and higher levels of student satisfaction with the 
collaborative learning techniques used in the course. 

Keywords: Learning Community; Collaborative Learning Engineering Orientation; Student Success; First-Year 
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INTRODUCTION 

study by Astin (1997) concluded that majoring in engineering correlated negatively with student 
satisfaction. Specifically, engineering majors were less satisfied with the quality of their instruction 
and overall college experience than students in other majors. In addition, engineering students felt 
more overwhelmed and depressed than their non-engineering peers. "Clearly, these findings indicate that the 
climate characterizing the typical institution with a strong emphasis on engineering is not ideal for student learning 
and personal development" (Astin, 1997, pp. 360-361). Other sources support Astin’s report, citing that engineering 
students are dissatisfied with their learning environments (Pascarella & Terenzini, 2005; Rompelman, 2000). 

Based on previous research, there appears to be a need for engineering educators to seek out and implement 
curricular strategies that encourage and support student learning. Education research indicates that in order to 
maximize student learning, educators must move away from more traditional lecture-based methods of teaching 
towards more engaging and student-centered approaches (Kuh, 1996; Pascarella, Springer, Terenzini, & Nora, 
1995). There is some evidence that this paradigm shift is occurring within engineering education. For example, 
ABET’s outcomes-based approach to evaluating engineering programs nationwide has shifted emphasis towards 
student learning and competencies (Volkwein et al., 2004; Engineering Accreditation Commission, 2007). This is in 
contrast to the previous ABET accreditation process, which was prescriptive and focused primarily on determining 
if programs contained sufficient numbers and types of courses, and one where student grades were used as the 
primary evidence of student learning (Yeargan, 2003). With the new criteria, programs must demonstrate that 
processes to continuously improve programs have been implemented. Additionally, programs must utilize 
assessment and evaluation processes to monitor and take steps towards improving outcomes at the student, course, 
and program levels. Educators cannot, however, rely solely on progress resulting in a response to accreditation 
requirements. Instead, educators must recognize their role in promoting student learning. 
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An overview of education research evidences the relationship between innovative teaching techniques and 
an increase in student learning and success. More specifically, students will learn more when the classroom is 
engaging, and when students have the opportunity to interact with their instructor and peers (Astin, 1993; 
Mazumder, 2010; Pascarella & Terenzini, 2005; Tinto, 1987). Student learning can be augmented when students are 
involved in the process of knowledge acquisition; for example, via active learning projects (Astin, 1985; Kuh, 
2003). This study sought to integrate findings from previous educational research in an effort to improve the 
success of engineering students at Oregon State University (OSU). In particular, the College of Engineering created 
a common first year schedule for a test cohort of first-year engineering students. 

The general engineering orientation course at OSU, Engineering 111 (ENGR 111), piloted a cohort model 
as a means for improving student success through interventions focused on increasing student learning, engagement, 
and persistence. One section of students (the test cohort) enrolled in ENGR 111 was registered into a common set of 
classes. The students were also registered in the same sections of all lectures, recitations, and laboratories. Success 
of students in the test cohort was compared with the success of all other students taking ENGR 111 in the same term 
as the control group. 

In this course, collaborative learning techniques were employed by the course instructor and teaching 
assistants. Testing was completed to determine whether or not these collaborative learning techniques were viewed 
more positively by cohort participants as compared to the control group participants. A review of the literature was 
used to guide the construction of the hypotheses tested in this study. 

REVIEW OF THE LITERATURE 

Three primary areas of previous research were used in developing the hypotheses for this study. These 
areas include: a) factors that affect student success, b) cohort-model learning communities, and c) collaborative 
learning. 

Factors Affecting Student Success 

Education research indicates that a key factor in college student success is student engagement (Astin, 
1977, 1985; Chickering & Gamson, 1987; Chickering & Reisser, 1993; Feldman & Newcomb, 1969; Kuh, 2001; 
Pace, 1979; Sanford, 1967). The definition of “student engagement” is based on two tenets. The first tenet is 
student-dependent: the amount of time and effort invested by a student in academic pursuits influences this student’s 
engagement. That is, student engagement is a measure of a student’s quantified commitment as well as the student’s 
quality of effort. For example, a student who commits to regularly reviewing course materials and lecture notes, to 
thoroughly completing weekly problem sets, and/or to studying with peers is considered to be engaged. The second 
tenet is institution-dependent: how an institution chooses to allocate resources and organize the curriculum 
influences student engagement. Institutions that create curricular opportunities to induce student participation will 
improve student success and subsequently, will graduate engaged students. Schools who offer student-centered 
approaches to learning will engage students more than peer institutions that cling to traditional pedagogies of 
teaching. Some examples of these approaches to learning include: learning communities, experiential learning 
opportunities, and collaborative learning environments (Kuh, 2003). 

A second factor that affects first-year student success is student interaction with others, including peers and 
faculty members. According to Astin (1997), “The student’s peer group is the single most potent source of 
influence on growth and development during the undergraduate years” (p. 398). The development of strong peer 
relationships is significant for both student social and academic support. Pascarella and Terenzini (2005) reviewed 
related evidence on students’ interactions with their peers and concluded that the frequency and the quality of 
student exchanges with peers were both positively associated with student success and persistence. At the same 
time, first-year student success was positively related to the quantity of student-faculty non-curricular interactions 
(those interactions taking place outside of the formal student-faculty relationship) and the regularity of student- 
faculty interactions to discuss academic matters (Pascarella & Terenzini, 2005; Cui et ah, 2011). Lundquist, 
Spalding, and Landrum (2003) found that the following faculty behaviors contributed to student success: a) 
accommodating of student needs, b) in-person approachability, and c) accessibility to students via telephone and 
email. 
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Models of Learning Communities 

A learning community can be defined as an intentional linkage or cluster of two or more courses, often 
around an interdisciplinary theme or problem, which enrolls a common cohort of students (Shapiro & Levine, 1999; 
Thompson, Oakes, & Bodner, 2005). Learning communities can range from scheduling groups of students to share 
two or more courses, to grouping students into a cohort and housing this cohort of students in the same residence 
hall. The most common form is the clustered course approach whereby students are linked together via common 
courses and a blocked schedule. In this model, the cohort is generally comprised of 20-30 students (Smith, 
MacGregor, Matthews, & Gabelnick, 2004). One course tends to be a writing course, and the cluster usually 
includes a weekly seminar. The weekly seminar plays an important role in helping students and faculty build 
connections in a smaller learning environment (Laufgraben, 2003). 

Fundamental outcomes of a learning community include: a) frequent faculty-student interaction, b) frequent 
student-student interaction, and c) time devoted to studying and collaborative learning opportunities (Laufgraben, 
2003). These intentionally-interactive small group environments assist the establishment of academic and social 
support networks inside and outside of the classroom (Collison, 1993; Davies et al., 2005; Laufgraben, 2003; 
MacGregor, Smith, Matthews, & Gabelnick, 1997). Collaborative learning, an integral part of the learning 
community, has also been shown to promote student success (Crissman, 2001; Gabelnick, Macgregor, Matthews, & 
Smith, 1999; Schroeder, 1994; Shapiro & Levine, 1999; Tinto & Goodsell Love, 1995; Tinto, Goodsell Love, & 
Russo, 1993). 

Education studies reveal that learning communities facilitate first year student adaptation to the college 
classroom environment (Castro-Cedeno, 2005; Kuh, 1996). The learning community environment provides an ideal 
setting in which to introduce students to the academic expectations and rigor of the college classroom, while student 
interaction with peers reinforces the attitudes essential to succeed as an engaged member of this community (Smith, 
MacGregor, Matthews, & Gabelnick, 2004). Student participants in the learning community are more likely to 
contribute in class discussions, to raise questions, and to seek an instructor’s help than non-participant peers. These 
students report greater satisfaction with courses and professors, and are more likely to participate in academic and 
social activities (Reumann-Moore, El-Haj, & Gold, 1997; Tinto, Goodsell Love, & Russo, 1993). 

Collaborative Learning 

There has been much published on the need for student-centered instruction. One approach to student- 
centered instruction is collaborative learning. Collaborative learning environments are characterized by students 
working together to reach a common academic goal (Ge, 2012). In this way, students master knowledge by 
constructing it. Collaborative learning techniques can utilize a variety of formats, technologies, as well as small and 
large groups of students. Examples of the collaborative learning techniques include, collaborative writing, group 
case study work, and group design projects (MacGregor, 1990). 

Proponents of collaborative learning maintain that the active exchange of ideas within small groups of 
students not only increases interest of the students, but also encourages critical thinking (Beckman, 1990). Johnson 
and Johnson (1986) suggest that cooperative teams function at higher levels of thought and retain information longer 
than students who work as individuals. The shared learning experienced via collaborative learning gives students the 
opportunity to engage in dialogue, take responsibility for their own learning, and thus develop critical thinking skills 
(Totten, Sills, Digby, & Russ, 1991). 

In addition, collaborative learning can also result in social benefits to the students, including developing 
students’ ability to view situations from others’ perspectives and the creation of social support systems (MacGregor, 
1990). Psychologically, collaborative learning has been shown to result in reduced classroom anxiety and increased 
student satisfaction. 
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Research Model 

This study sought to determine whether or not student participation in a learning cohort positively affected 
student success in a course where collaborative learning techniques were applied. Post-hoc analysis was also 
completed to study whether or not student demographics could account for any observed differences. The research 
model developed to guide this study is shown in Figure 1. 



Figure 1: Research Model 


Eight different hypotheses were tested in the study and are summarized in Table 1. Student success was 
measured by evaluating student learning, engagement, and persistence. Student learning was evaluated through an 
assessment of gains in knowledge and skills and through assessing student attitudes regarding engineering as a 
discipline. Gains in knowledge were measured through the analysis of student coursework in ENGR 111 and 
cumulative OSU GPAs. Gains in skills were measured using an assessment of the development of study skills. 
Student engagement was measured through an evaluation of participation in ENGR 111, as measured by attendance 
and by assessing attitudes towards working with others, as well as peer-to-peer interactions. Persistence was 
measured by looking at whether or not students were enrolled in the engineering program at two points in time - one 
term after the intervention and one year after the intervention. Post hoc testing was completed to determine if 
uncontrolled inputs (student demographics or undergraduate teaching assistant) could provide alternate explanations 
for any identified, significant differences. 


Table 1: Research Hypotheses for ENGR 111 Cohort Study 


Input Variable 

Null Hypothesis 

Alternate Hypothesis 

Knowledge (K) 

H lo Student participation in a learning cohort is 
not correlated to student GPA and course 
performance. 

H la Student participation in a learning cohort is 
positively correlated to student GPA and 
course performance. 

Skills (S) 

H 2o Student participation in a learning cohort is 
not correlated to effective study skills. 

H 2a Student participation in a learning cohort is 
positively correlated to effective study skills. 

Attitudes (A) 

H 3o Student participation in a learning cohort is 
not correlated to student attitudes towards 
the discipline of engineering. 

H 3a Student participation in a learning cohort is 
positively correlated to student attitudes 
towards the discipline of engineering. 

Engagement (E) 

H 4o Student participation in a learning cohort is 
not correlated to attendance scores in 
ENGR 111. 

H So Student participation in a learning cohort is 
not correlated to student attitudes towards 
working with others. 

H 6o Student participation in a learning cohort is 
not correlated to the level of student to 
student interaction. 

H 4a Student participation in a learning cohort is 
positively correlated to higher attendance 
scores in ENGR 111. 

H Sa Student participation in a learning cohort is 
positively correlated to student attitudes 
towards working with others. 

H 6a Student participation in a learning cohort is 
positively correlated to an increased level of 
student to student interaction. 
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Table 1 cont. 


Collaborative 
Learning (CL) 

H 7o Student participation in a learning cohort is 
not correlated to student evaluation of the 
collaborative learning techniques. 

H 7a Student participation in a learning cohort is 
positively correlated to student evaluation of 
collaborative learning techniques. 

Persistence (P) 

H 8o Student participation in a learning cohort is 
not correlated to continued enrollment in 
the engineering program. 

H Sa Student participation in a learning cohort is 
positively correlated to continued enrollment 
in the engineering program. 


Course Description 

Engineering 111, Engineering Orientation I, is a ten-week, 3-credit first-year course designed for pre¬ 
engineering students who have not yet chosen a particular engineering major. This course is intended to give 
students a broad introduction to the profession and disciplines of engineering. Emphasis is placed on the application 
of engineering and design concepts to generate solutions to engineering-based problems. There are no pre-requisites 
for the course. Students declaring the pre-engineering major of “General Engineering” at OSU are required to take 
Engineering 111. The number of students who completed the fall term course during the study was 123 students. 
Student grades were assigned based on attendance, homework, a team design project, a midterm exam, and a final 
exam. Each of these elements was assessed out of 100 points and accounted for 20% of the student’s final course 
grade. 


The Engineering 111 curriculum was delivered via three teaching environments: a) lecture, b) laboratory, 
and c) recitation. Students attended the lectures, labs, and recitations weekly. The 50-minute lecture included guest 
speakers from different engineering disciplines as well as instruction on engineering design and project 
management. Lectures were either delivered by or facilitated by the course instructor. While all students attended 
the lecture together, students were divided into smaller groups (n = 18 to 23) and met twice a week for a two-hour 
laboratory and a two-hour recitation. Each laboratory session focused on experiments from the various engineering 
disciplines, in conjunction with the discipline presented during lecture that particular week. The recitations covered 
various academic success topics, engineering ethics and professionalism, and technical topics, including dimensional 
analysis and correctly using of significant digits. Recitation topics and some of the learning techniques, including 
collaborative learning techniques, used through the term are summarized in Table 2. 


Table 2: Learning 

Activities used during ENGR 111 Recitations 

Week Recitation Topic 

Sample of Learning Activities 

1 - Managing the transition to OSU 

One minute paper on “First impressions of college” 

High school vs. university differences group activity 

Develop questions for student panel for lecture in week 2 

Review and discussion of laboratory notebook requirements 

2 - Classroom success 

One minute paper on “How to get the most out of my classes” 

Library tour 

Class note comparison activity 

3 - Choosing a major and selecting classes & 
representing technical information 

One minute paper on “Questions I have about engineering majors” 
Academic advisor presentation and Q&A 

Presentation on using charts and graphs to present technical information 

4 - Learning styles & estimates and error 
approximations 

One minute paper on “How I learn best” 

VARK learning style assessment and case study 

Estimates and error approximation worksheet 

5 - Managing your OSU workload 

One minute paper on “Creating a plan for midterm success” 

Academic Success Center presentation 

Discussion of available student services 

6 - Balancing it all & measurement units and 
conversions 

Self-reflection of student midterm performance 

One minute paper on “Coping with stress” 

Peer Health Educators talk on balance and stress management 

Units and conversions worksheet 

7 - Working on Engineering teams 

One minute paper on “Making teams successful” 

Meyers Briggs Type Indicator assessment and discussion 

Discussion of course planning for next term 

Diversity of Species activity 
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Table 2 cont. 


8 - Team Projects 

Check in on team project status 

Time to work on team projects 

9 

Thanksgiving break 

10 - Engineering ethics and professionalism & 
wrap up 

Engineering ethics presentation 

Discussion and completion of engineering ethics worksheet on Bhopal 
case study 


Six undergraduate teaching assistants were employed to teach the laboratory and recitation sections of this 
course. Research supports the decision to use undergraduate teaching assistants, as peer instruction has been found 
to be a more effective learning tool than instructor-led learning (McCreary Golde, & Koeskie, 2006). Five of the six 
teaching assistants were senior Industrial Engineering students; the sixth assistant was a senior in Chemical 
Engineering. The Industrial Engineering students were all male, while the one Chemical Engineering student was 
female. Undergraduate teaching assistants were recruited for this role based on their ability to communicate 
effectively and lead peer groups. 

Each teaching assistant was responsible for teaching the same group of students for the duration of the 
term. In addition to leading the weekly laboratory and recitation sessions, each teaching assistant was responsible 
for taking attendance and grading homework and tests. Teaching assistants were trained before the start of the term 
on relevant topics including professionalism, teaching methodology, and diversity awareness. Specific lesson plans 
were developed and used for both recitation and laboratory sessions each week. During the term, teaching assistants 
trained weekly with the instructor and a graduate student in the College of Education’s College Student Services 
Administration program on the materials that would be covered in the upcoming week’s laboratory and recitation. 

Research Variables 

Research by Hinton (1993) concludes that classroom instruments that are based on student perceptions of 
the efficacy of teaching and learning methods are both valid and reliable. Such instruments provide robust 
information on student perceptions of learning-more so than typical course evaluations, which show little useful 
information related to teaching pedagogy or student learning. More specifically, traditional course evaluations serve 
to tell instructors what students “liked” about the course, but tells little about what the student gained from their 
participation in a class (Seymour, Wiese, Hunter, & Daffinrud, 2000). 

An existing online survey, the Student Assessment of Learning Gains (SALG), was used to assess student 
skills, attitudes engagement, and student perceptions of the collaborative learning elements used in ENGR 111. The 
SALG was developed by Seymour, Wiese, Hunter, & Daffinrud at the University of Colorado at Boulder (2000). 
This instrument measures student perceptions of learning gains made in relation to specific components of a course. 
More specifically, the following types of gains can be explored: skill development, cognition, attitudes, and learning 
retention. Thus, the SALG instrument can be used to provide feedback on student learning gains in relationship to a 
specific course or series of courses. The hosting website allows customization of the instrument. The standard set 
of items can be modified, added, or deleted. The instrument’s website is also used for survey administration and to 
store student responses. Data from completed surveys can be downloaded into other software packages for 
additional manipulation and analysis (Seymour, 2006). 

The SALG survey, customized for ENGR 111, contained 56 items divided into six sections. A five-point 
Likert scale was used for all items. The first section focused on the impact of different course elements on student 
learning; e.g., class activities, homework assignments, exams, textbooks. The second section of the survey centered 
on gains in engineering-specific knowledge; e.g., understanding of unit conversion and dimensional analysis, 
understanding of the engineering design process, etc. The third section included items related to the development of 
study skills; e.g., managing time, taking effective notes, using campus resources, etc. The fourth section included 
items relating to broader capabilities related to success in academia and in engineering; e.g., feeling comfortable 
with complex ideas, ability to think through problems, etc. The items in the fifth section probed whether or not 
students felt that elements of the course would be pertinent to other classes or in other aspects of the student’s life; 
e.g., study skills, working effectively in groups, problem solving abilities, etc. 
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The sixth and final section consisted of items asking students to evaluate how strongly students felt about 
educational goals, such as graduating from college and completing studies in engineering. This section also 
included items asking students to assess whether or not students had developed relationships with faculty and peers 
to help them be successful. A summary of the survey items used for to measure skills (S), attitudes (A), student 
engagement (E2 and E3), and collaborative learning (CL) are summarized in Table 3. 


Table 3: SALG Items Used to Assess Student Success 


Variable Name 

Variable Description 
(Cronbach’s Alpha) 

SALG Items 

Skills (S) 

Ability to study and use the 
resources available on campus 
effectively 

0.891 

Q3. "How much has this class added to your skills in each of the following? ” 

Q3.1: Studying effectively 

Q3.2: Taking effective notes 

Q3.3: Understanding my learning style 

Q3.4: Managing my time effectively 

Q3.5: Using campus resources 

Q3.6: Understanding what is expected of me as a university student 

Attitude (A) 

Attitudes toward the 

engineering discipline 

0.873 

Q4. "To what extent did you make gains in any of the following as a result of what you did 
in this class? ” 

Q4.5: Understanding the value of engineering in solving real world issues 

Q4.6: Appreciating engineering as a discipline 

Q4.7: Enthusiasm for engineering 

Q4.8: Confidence in my ability to be successful in my engineering studies 

Engagement (E2) 

Attitude toward working with 
others 

0.868 

Q3. “How much has this class added to your skills in each of the following? ” 

Q3.7: Working effectively with others 

Q5. “How much of the following do you think you will remember and carry with you into 
other classes or aspects of your life? ” 

Q5.5: Working effectively with others to study 

Q5.6: Working effectively in groups on projects 

Engagement (E3) 

Peer-to-peer interactions 

0.823 

Q1H. "How much did each of the following aspects of the class help your learning? 

Q1H2: Working with peers during labs and recitation 

Q1H3: Working with peers outside of class 

Q1H4: Studying with my peers 

Collaborative Learning (CL) 

Course activities including 
collaborative elements 

0.813 

Ql. How much did each of the following aspects of the class help your teaming? 

Q1.D2: Recitation activities 

Q1.D3: Lab activities 

Q1.E3: The team design project 

Q1.H1: Interactions with my lab/recitation TA 

Q1.H2: Working with peers during labs and recitations 


The remaining outcome variables were not measured using the SALG. The variable, knowledge (K), was 
measured using two different data sets. First, cumulative OSU GPA’s at the end of the second term of enrollment at 
OSU were collected. Second, detailed student coursework performance data was collected from the instructor’s 
course records. Scores for eight homework assignments, a midterm exam, a final exam, and a design project were 
used to provide an independent measure of student knowledge. Student attendance scores were also collected from 
the instructor’s course records. This data was used to provide another measure of student engagement (El). The 
final variable, not measured using the SALG was persistence. Persistence (P) was measured using student 
enrollment status one term and again one year after the intervention. Student demographic data from the university 
data warehouse was also collected for each student enrolled in ENGR 111. 

Participants 

Institutional Review Board (IRB) permission was sought and obtained for this study. Students were asked 
to join the learning community cohort during the summer orientation and registration session, after determining that 
students were also planning to take Calculus (MTH 251) and first-year Chemistry (CHEM 201). The only incentive 
for joining the cohort was being able to take Public Speaking (COMM 111) in the first term of their first year. This 
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class is typically filled by upper division students, making it difficult for first year students (who register last) to find 
an open slot. In total, students registered in the test cohort shared four courses in the Fall term, including first year 
Chemistry (CHEM 201), Public Speaking (COMM 111), Calculus (MTH 251), and Engineering Orientation (ENGR 
111). Control group student schedules included Engineering Orientation (ENGR 111) and other courses typical to a 
first-year engineering student; e.g., math, chemistry, writing, and/or health. No attempts were made to organize 
control group student schedules. 

All ENGR 111 students were given the opportunity to complete the SALG survey. Participation was 
voluntary. The survey was administered in recitations during the last week of the academic term. The 
undergraduate teaching assistants distributed instructions for completing the on-line survey and read a script 
informing students about the survey, along with their rights as study participants. Students were permitted to ask 
questions prior to determining whether or not to participate. Using a course ID and password provided by the 
teaching assistants, students logged in and completed the online survey. This process required approximately 10 to 
15 minutes of time. 

A total of 96 students chose to complete the SALG. This corresponds to a response rate of 78%. In total, 
20 students were registered into the test cohort. All test cohort participants were male. Of the 20 registered in the 
test cohort, 13 completed the SALG. Thus, the remaining 83 responses represented control group participants from 
the five other sections. SALG data was collected by recitation, enabling comparisons between the test cohort and 
either the entire group of control participants or comparison to individual sections within the control group. 

To maintain anonymity, demographic data specific to the group of students who chose to complete the 
SALG was not collected. Demographic data for all ENGR 111 students was available, however, from the university 
data warehouse. 81% of the students were male and 19% were female. White, non-Hispanic students made up 75% 
of the student population, while 15% were Asian. The third group consisted of ethnicities typically 
underrepresented in engineering including Hispanic, African-American, and Native Indian students. This combined 
group of students made up 10% of the ENGR 111 student population. Students from rural locations made up 40% 
of the student population in ENGR 111, while the remaining 60% came from urban areas. 

RESULTS 

Data analysis for this study was conducted using SPSS 15.0. Three different data sources were used in 
completing the analysis and in testing the seven research hypotheses: the SALG responses, the university data 
warehouse (cumulative GPAs, demographic data, and student enrollments), and the instructor’s course records. The 
analyses used for the various data sets are described and the results from these analyses are summarized in this 
section. 


The number of responses, means, and standard deviations for each of the five outcomes variables measured 
using the SALG, as well as student assessment of the collaborative learning elements in the course are summarized 
in Table 4. The number of SALG responses used for determining means and standard deviations varied slightly due 
to incomplete surveys. If a student did not complete all items for one of the variables, the data for that variable from 
that student was not used in the analysis. Reliability coefficients (Cronbach’s Alpha) were calculated for the five 
variables created from a subset of SALG items (S, A, E2, E3, and CL). Cronbach alpha values for each of these 
variables are included in Table 3. A Cronbach’s alpha of at least 0.7 is considered to provide evidence of a reliable 
scale. Based on this criterion, reliabilities were acceptable for all five variables constructed from SALG responses. 
Summary statistics for the outcome variables used to measure student knowledge (Kl, K2, K3, K4, and K5) and 
student engagement as measured by attendance scores (El) are also included in Table 4. The maximum possible 
GPA at OSU is a 4.00. The maximum possible score for each aspect of the course grade (homework, midterm 
exam, final exam, attendance, and design project) was 100. 
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Table 4: Descriptive Statistics for Outcome Variables and Collaborative Learning 


Variable 

11 

Mean 

Std Dev 

S (Study skills) 

97 

2.85 

0.797 

A (Attitudes towards the discipline of engineering) 

95 

3.11 

0.854 

E1 (Engagement as measured by attendance scores) 

123 

98.21 

4.66 

E2 (Attitudes towards working with others) 

97 

3.28 

0.866 

E3 (Peer-to-peer interactions) 

97 

3.40 

0.830 

CL (Collaborative Learning) 

97 

3.33 

0.760 

K1 (Knowledge as measured by cumulative OSU GPA) 

118 

2.94 

0.759 

K2 (Knowledge as measured by homework scores) 

123 

86.39 

12.48 

K3 (Knowledge as measured by midterm exam scores) 

123 

83.09 

8.65 

K4 (Knowledge as measured by final exam scores) 

123 

81.86 

10.54 

K5 (Knowledge as measured by design project scores) 

123 

92.32 

3.99 


Having determined that the SALG responses demonstrated acceptable levels of reliability, additional 
analyses were completed to determine the appropriate tests to use for the research hypotheses. Results from 
Levene’s test for equality of variances indicated that variances were not significantly different between the test 
cohort and control group for the variables S, A, E2, E3, CL, and OSU GPA. Evidence of non-equal variances was 
found, however, for homework (K2) and attendance scores (El). The normality of the data was assessed for each 
variable using a one-sample Kolmogorov-Smirnov procedure and by evaluating Q-Q plots. There was evidence of 
non-normality for all variables created from the SALG data and for homework (K2) and attendance scores (El). 
OSU GPA data, midterm exam scores, final exam scores, and design project scores were normally distributed. Non- 
parametric testing was used for all subsequent analyses, except in evaluating differences for OSU GPA’s, midterm 
exam scores, final exam scores, and design project scores. 

One sided tests were used for each hypothesis based on the evidence found during literature review 
supporting a positive relationship between learning communities as well as collaborative learning techniques and 
student success. Decisions on the statistical significance of results were made using an alpha (a) of 0.05. Where 
significant differences were identified, post-hoc testing was conducted to determine the location of these differences. 
Post-hoc analysis of significant differences between groups was completed using Dunnett’s test. Since the test 
cohort was compared against each section within the control group, an alpha of 0.01 was used to determine 
statistical significance, since five comparisons were performed. The results of the statistical analyses completed for 
each variable are summarized by hypothesis in Table 5. The type of test completed, the test statistic, and the p-value 
are included. 


Table 5: Hypotheses Testing Results for H t - H 7 


Hypothesis 

Variable 

Test type 

Test Statistic* 

p-value 

Hi 

K1 (Cumulative OSU GPA) 

ANOVA 

1.12 

0.352 

K2 (Homework) 

Kruskal Wallis 

17.86 

0.003 

K3 (Midterm exam) 

ANOVA 

0.95 

0.453 

K4 (Final exam) 

ANOVA 

1.15 

0.339 

K5 (Design project) 

ANOVA 

6.20 

0.000 

h 2 

S (Study skills) 

Kruskal Wallis 

4.78 

0.311 

h 3 

A (Discipline of engineering) 

Kruskal Wallis 

8.36 

0.079 

h 4 

El (Attendance) 

Kruskal Wallis 

16.71 

0.005 

H, 

E2 (Working with others) 

Kruskal Wallis 

1.61 

0.807 

h 6 

E3 (Peer-to-peer interactions) 

Kruskal Wallis 

2.99 

0.559 

h 7 

CL (Collaborative learning) 

Kruskal Wallis 

13.99 

0.007 


* F-statistic for ANOVA; Chi-Square for Kruskal Wallis 


Four statistically significant relationships were identified. Based on this analysis, there is evidence for a 
relationship between which section students were enrolled in and two measures of knowledge acquisition 
(homework scores and design project scores), between section enrollment and one measure of student engagement 
(attendance scores) and between section enrollment and student evaluations of the impact of collaborative learning 
activities. Since different teaching assistants were used for each section of students, post hoc testing was completed 
to identify whether or not the test cohort differed from any specific sections within the control group. 
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Post hoc testing of the design team project scores found that while differences existed between sections, 
these differences could not be explained by participation in the test cohort. Two of the sections within the control 
group performed worse on the design project than the remaining sections in the control group and the test cohort, but 
the test cohort performed similarly to all other sections. After taking into account the corrected significance level 
for multiple comparisons, it was not possible to detect significant differences by section for homework scores or 
student attendance. Results from post hoc testing did, however, indicate that the test cohort rated the impact of 
collaborative learning activities significantly higher than one of the five sections in the control group. 

Since participation in the cohort could not fully explain the observed differences, further analysis was 
completed to determine whether or not student characteristics, such as gender or ethnicity, could explain the 
observed differences in homework scores, term project scores, or in student attendance. Using a Mann-Whitney test 
(nonparametric test for two independent samples), a significant difference (p = 0.026) was identified for homework 
scores, when gender was taken into account. In particular, female students had higher homework scores than their 
male counterparts. ANOVA was used to test for significant differences in term project scores, accounting for 
gender. A significant difference (p < 0.000) was found. Gender was not a significant factor in attendance scores. 

A second set of post-hoc tests was conducted to determine whether or not ethnicity could potentially 
explain differences in homework scores, design project scores or student attendance. When the scores of students 
who are not traditionally represented in engineering (Native American, Hispanic, Latino, Alaskan Native, African 
American) were compared with the remaining students (White, Euro American, Non-Hispanic, Asian, and Asian 
American), students from the underrepresented groups had significantly lower homework and attendance scores (p = 
0.003 and p = 0.009). Ethnicity, however, did not appear to be a significant factor in term project scores. 

The final hypothesis, H 8 , was tested using the persistence data collected both one term after and again one 
year after the intervention. A cross tabulation of enrollment data is summarized in Table 6. The percentage of 
students leaving engineering for the test cohort was 10% at both times. The percentage of students leaving 
engineering from the control group was 19% one term after the intervention and 21% one year after the intervention. 
Fisher’s Exact Test was performed to test for differences. No significant difference was found. 

Table 6: Cross-Tabulation of Enrollment Data One Term and One Year after the Intervention 


One term after intervention 

Test 

Cohort 

Control 

Group 

Transferred out of engineering 

2 

17 

Remaining in engineering 

18 

72 


One year after intervention 

Test 

Cohort 

Control 

Group 

Transferred out of engineering 

2 

19 

Remaining in engineering 

18 

70 


DISCUSSION 

This study investigated whether student participation in a learning community affects student success as 
measured by learning, engagement, and persistence. The study results only identified one significant difference 
linked to participation in the learning community. Education literature indicates that more structured learning 
communities are tied more closely to student gains in learning (Gabelnick, MacGregor, Matthews, & Smith, 1999). 
While ENGR 111 cohort participants did share a common first term schedule, the cohort was not represented in a 
formalized way to this student population. The rationale for the common schedule was not discussed with the cohort 
group, which may have limited the impact of the intervention, the common set of classes on these students. 

Within the education literature there is also some discussion on the importance of the bridging or synthesis 
course as part of the learning community model. This bridging course is often designed to help students to make 
connections across different academic disciplines, supporting increased levels of student learning and engagement 
(Davies, Ramsay, Lindfield, & Couperwaite, 2005; Gabelnick, MacGregor, Matthews, & Smith, 1999). The ENGR 
111 cohort model did not include a separate course to serve this purpose. The topics covered in the ENGR 111 
laboratories and recitation, while touching on a variety of science and mathematical concepts did not specifically 
connect with content in other courses being taken by the participants; e.g. chemistry or calculus. It is possible that 
without this element, the full benefit of the learning community model is not realized. 
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Another factor that may have impacted the results of this study is the individual undergraduate teaching 
assistants’ teaching styles. The research of Gutwill and Seymour (1999) indicates that peer instructors can play a 
significant role in student learning. Two of the sections of students in the control group consistently scored course 
elements lower than the remaining four sections, including the test cohort. Students in these sections rated recitation 
activities, lab activities, the grading system, and laboratory handouts as less helpful than other students. 
Interestingly, these students also evaluated their interactions with the undergraduate TA lower, indicating that these 
students felt less positive about the role of the undergraduate TA in their learning. Observations of the six 
undergraduate teaching assistants in laboratory as well as recitation settings by the graduate teaching assistant and 
the instructor provided some evidence of differences in teaching styles. For example, some TAs used a lecture- 
based approach while other TAs included group discussion and other less formal methods of instruction. It is 
possible that these differences in teaching style are linked with student gains in learning as it has been evidenced that 
student-centered approaches to teaching, when compared to more traditional professor-centered methods, will 
facilitate student learning and success (Kuh, 1996; Pascarella, Springer, Terenzini, & Nora, 1995). Due to the nature 
of the data, these results cannot be used to show causation, but the results do suggest that the undergraduate teaching 
assistant may play a significant role in student learning. 

Based on the results of the post-hoc analysis, there was some evidence that demographic factors (gender 
and ethnicity) may be tied to student success, both in terms of knowledge and engagement. Since no members of 
the cohort group were female and only one member of the cohort was a member of a traditionally underrepresented 
group within engineering, it was not possible to test for interactions between these demographic factors, 
participation in the cohort, and student success. However, the findings do indicate the need for additional study. A 
larger study would be needed to determine if there are indeed interactions between demographic factors, cohort 
participation, and the various measures of student success. 

Overall, students rated the recitation and laboratory course components as being highly supportive of their 
learning. Student feedback regarding the non-traditional and small group interactions in the course was positive. 
The mean score of these items was higher than 3.0 on a 5.0 scale. A score of 3.0 corresponded to a qualitative 
statement of moderate help; whereas a score of 4.0 corresponded to a qualitative rating of much help. So while it 
was not possible to directly correlate participation in the test cohort with gains in student learning, overall ratings 
indicate that students, both in and outside of the test cohort, felt positively about the collaborative learning elements 
in the course. 

Participants in the test cohort rated the collaborative learning elements of the course as being more helpful 
to their learning than students in the control group. It is possible that student cohort participants, given the increased 
interaction time, were able to collaborate more effectively with their group members during recitations and 
laboratories, resulting in their increased rating for the role of collaborative learning on their own personal learning 
gains. 


While the research hypotheses developed for this study were addressed, at least in a specific setting, 
additional research is needed to generalize these findings. The impact of institutional-specific characteristics, for 
example, could not be tested for, but is an interesting factor to explore in a multi-institutional study or through 
replicating this study using other institutions. In addition, the results of this study have also identified two potential 
questions for future research. While this study only considered a single test group, there was not strong evidence to 
support a positive correlation between participation in the learning community model used (a common core of 
classes) and increased student success. One question to be answered is what elements are required for a learning 
community of engineering students to support student success? Are, for example, synthesis or bridge courses 
critical in realizing the full benefits of learning communities? Further, the cohorts for this study were selected based 
on the common schedules rather than using a protocol for assignment. While there was no evidence to suggest that 
common schedules correlated to certain student characteristics, future studies could benefit from a more deliberate 
selection of students into cohorts. 

Second, how significant is the role of teaching styles by peer instructors in rendering collaborative learning 
elements and learning communities more or less effective? Conducting studies within an actual educational setting 
presents a variety of challenges. While all efforts were taken to train the peer instructors for this study in exactly the 
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same way, there was evidence that peer instructors created different learning environments and these environments 
appeared to be correlated with student performance. Additional testing must be done to further understand this 
relationship and may be useful in developing future training for peer instructors. 

Education literature shows that students who have confidence in their abilities within their academic 
discipline will persist at higher rates than those who do not (Pascarella & Terenzini, 2005). It is evidenced in 
literature and benchmarking that engineering schools across the United States struggle to retain their students to 
graduation (Castro-Cedeno, 2005). This study suggests that collaborative learning, and to a lesser extent, learning 
communities, may play a role in engineering student success. While there was not any indication in previous 
research that learning communities and collaborative learning are more or less effective in different disciplines, the 
results of this study do suggest that additional research is needed to more fully characterize the potential impact of 
these techniques on engineering student success. These results, along with recent academic conversations related to 
engineering curriculum development (Saunders et al., 2013), highlight the importance of deliberate design work to 
create compelling opportunities for student learning and thinking outside of traditional means for engaging 
engineering students in their own learning. 
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